Supplying content aware photo filters

ABSTRACT

A server includes a photo filter module with instructions executed by a processor to identify when a client device captures a photograph. Photograph filters are selected based upon attributes of the client device and attributes of the photograph. The photograph filters are supplied to the client device.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of and claims the benefit of priorityto U.S. patent application Ser. No. 16/175,288, filed Nov. 1, 2018,which is a continuation of and claims the benefit of priority of U.S.patent application Ser. No. 15/224,262, filed Jul. 29, 2016, which is acontinuation of U.S. patent application Ser. No. 14/977,380 filed onDec. 21, 2015, which is a continuation of U.S. patent application Ser.No. 14/325,270, filed Jul. 7, 2014 (now U.S. Pat. No. 9,225,897), thecontents of which are incorporated herein by reference.

FIELD OF THE INVENTION

This invention relates generally to photographs taken by a mobile deviceoperative in a networked environment. More particularly, this inventionrelates to supplying such a mobile device with content aware photofilters.

BACKGROUND OF THE INVENTION

The number of digital photographs taken with mobile wireless devices isincreasingly outnumbering photographs taken with dedicated digital andfilm based cameras. Thus, there are growing needs to improve theexperience associated with mobile wireless digital photography.

SUMMARY OF THE INVENTION

A server includes a photo filter module with instructions executed by aprocessor to identify when a client device captures a photograph.Photograph filters are selected based upon attributes of the clientdevice and attributes of the photograph. The photograph filters aresupplied to the client device.

BRIEF DESCRIPTION OF THE FIGURES

The invention is more fully appreciated in connection with the followingdetailed description taken in conjunction with the accompanyingdrawings, in which:

FIG. 1 illustrates an electronic device utilized in accordance with anembodiment of the invention.

FIG. 2 illustrates a networked system utilized in accordance with anembodiment of the invention.

FIG. 3 illustrates processing operations associated with an embodimentof the invention.

FIG. 4 illustrates a photograph taken by a digital mobile device.

FIG. 5 illustrates a general filter applied to the photograph.

FIG. 6 illustrates a feature specific filter applied to the photograph.

FIG. 7 illustrates a different feature specific filter with a brandedelement applied to the photograph.

Like reference numerals refer to corresponding parts throughout theseveral views of the drawings.

DETAILED DESCRIPTION OF THE INVENTION

FIG. 1 illustrates an electronic device 100 utilized in accordance withan embodiment of the invention. In one embodiment, the electronic device100 is a Smartphone with a processor 102 in communication with a memory104. The processor 102 may be a central processing unit and/or agraphics processing unit. The memory 104 is a combination of flashmemory and random access memory. The memory 104 stores a photocontroller 106. The photo controller 106 includes executableinstructions to coordinate the capture, display and archiving of digitalphotographs. The photo controller 106 may include photo filterprocessing disclosed herein, which augments or replaces such photofilter processing that is described below in connection with a serverbased photo filter module.

The processor 102 is also coupled to image sensors 115. The imagesensors 115 may be known digital image sensors, such as charge coupleddevices. The image sensors capture visual media, which is presented ondisplay 116, as coordinated by the photo controller 106.

A touch controller 118 is connected to the display 116 and the processor102. The touch controller 118 is responsive to haptic signals applied tothe display 116. In one embodiment, the photo controller 106 monitorssignals from the touch controller 118 to coordinate the capture, displayand archiving of digital photographs. The electronic device 100 may alsoinclude other components commonly associated with a Smartphone, such asa wireless signal processor 120 to support wireless communications, apower control circuit 122 and a global positioning system processor 124.

FIG. 2 illustrates a system 200 configured in accordance with anembodiment of the invention. The system 200 includes a set of clientdevices 100_1 through 100_N. The client devices 100 are connected to anetwork 206, which is any combination of wireless and wired networkcommunication devices. A server 204 is also connected to the network206. The server 204 includes standard components, such as a centralprocessing unit 210 and input/output devices 212 connected via a bus214. The input/output devices 212 may include a keyboard, mouse, displayand the like. A network interface circuit 216 is also connected to thebus 214 to provide connectivity to network 206. A memory 220 is alsoconnected to the bus 214. The memory 220 includes modules withexecutable instructions, such as a photo filter module 222. The photofilter module 222 implements photo evaluation and filter selectionoperations, as discussed below.

FIG. 3 illustrates processing operations associated with an embodimentof the invention. The operations are performed by the photo filtermodule 222 of server 204 in combination with one or more client devices100. Initially, the photo filter module 222 serves a photo prompt 300.For example, the photo filter module 222 may form a segment of a networkexecuted application that coordinates taking photographs and appendingmessages to such photographs for delivery from one user to another. Inthis context, client 100_1 accesses the photo filter module 222 overnetwork 206 to activate the application, which serves the photo promptto the client 100_1. A user at the client 100_1 takes a photo 302.

FIG. 4 illustrates client device 100_1 with a display 400 that presentsa photo prompt 402. Activation of the photo prompt 402 results in apicture 404. The photo filter module 222 monitors the client deviceactivity to determine if a photo is taken 304. If so, the attributes ofthe photograph and client device are evaluated 305. Photo filters areselected and supplied 306 based upon the evaluation.

By way of example, the attributes of the client device may includegeolocation of the client device, which is collected from the GPSprocessor 124. The geolocation may be used to designate photo filtersrelevant to the geolocation. For example, if the geolocation isproximate to a beach, then photo filters to augment a beach setting(e.g., a color filter for water, sand and/or sky) may be supplied. Thegeolocation may be used to select a filter with a brand associated withan establishment proximate to the geolocation. For example, a restaurantor store may sponsor a photo filter that includes a brand associatedwith the restaurant or store. In this case, in addition to the brand,the photo filter may include other indicia associated with therestaurant (e.g., an image of a hamburger or taco) or store (e.g., animage of a surf board or sun glasses).

The attributes associated with the client device may include establishedpreferences associated with the client device. The establishedpreferences may be defined by explicitly stated preferences supplied bya user. Alternately, the established preferences may be derived fromprior use patterns. For example, explicitly stated or derivedpreferences may indicate that photo filters with a temperature overlay,date and/or time overlay be supplied.

The attributes of the photograph may include the physical environmentcaptured in the photograph. For example, the photograph may be evaluatedto identify an urban setting, a rural setting, a sunset a seascape andthe like. Filters applicable to the physical environment may then besupplied.

The attributes of the photograph may include an object depicted in thephotograph. For example, the evaluation may identify a building, abuilding feature (e.g., door or roof), a flower, an individual, ananimal and the like. Filters applicable to such objects may then besupplied.

The next operation of FIG. 3 is to apply the photo filters 308. Forexample, a swipe across the display of a client device 100_1 may cause aphoto filter to slide across the original photo. FIG. 5 illustrates theresult of a first swipe motion, which results in a darkening filter 500being applied to the original photo. Another swipe motion may result inanother filter being presented. For example, FIG. 6 illustrates theresult of a second swipe motion, which results in an object specificfilter 600 being presented. In this case, the object specific filter 600relates to the identification of a door in the photo. The identificationof the door may result in the supply of a variety of filters fordifferent door colors. Another swipe of the display may result in stillanother filter, such as shown in FIG. 7. The filter of FIG. 7 includesan object specific filter 700, in this case for a roof of a building.The filter also includes a brand component 702. This filter alsoincludes an overlay of the temperature 704 when the photo was taken. Atime overlay 706 and date overlay 708 are also supplied.

Returning to FIG. 3, the next operation is to select a photo filter 310.Selection of a photo filter may include selection of one or moreavailable filters. The photo may then be saved with the applicablefilter or filters. The photo and filter may also be sent to another user312. In this case, the server 204 routes 314 the photo to another client100_2, which displays the photo with the filter 316.

Photograph filters may also be selected based upon popular filters.Branded filters may be supplied based upon an auction mechanism. Forexample, vendors may bid on photo filters to be supplied based uponcharacteristics of a user, location of a user, content of a photographand the like.

An embodiment of the present invention relates to a computer storageproduct with a non-transitory computer readable storage medium havingcomputer code thereon for performing various computer-implementedoperations. The media and computer code may be those specially designedand constructed for the purposes of the present invention, or they maybe of the kind well known and available to those having skill in thecomputer software arts. Examples of computer-readable media include, butare not limited to: magnetic media, optical media, magneto-optical mediaand hardware devices that are specially configured to store and executeprogram code, such as application-specific integrated circuits(“ASICs”), programmable logic devices (“PLDs”) and ROM and RAM devices.Examples of computer code include machine code, such as produced by acompiler, and files containing higher-level code that are executed by acomputer using an interpreter. For example, an embodiment of theinvention may be implemented using JAVA®, C++, or other object-orientedprogramming language and development tools. Another embodiment of theinvention may be implemented in hardwired circuitry in place of, or incombination with, machine-executable software instructions.

The foregoing description, for purposes of explanation, used specificnomenclature to provide a thorough understanding of the invention.However, it will be apparent to one skilled in the art that specificdetails are not required in order to practice the invention. Thus, theforegoing descriptions of specific embodiments of the invention arepresented for purposes of illustration and description. They are notintended to be exhaustive or to limit the invention to the precise formsdisclosed; obviously, many modifications and variations are possible inview of the above teachings. The embodiments were chosen and describedin order to best explain the principles of the invention and itspractical applications, they thereby enable others skilled in the art tobest utilize the invention and various embodiments with variousmodifications as are suited to the particular use contemplated. It isintended that the following claims and their equivalents define thescope of the invention.

The invention claimed is:
 1. A system comprising: a processor configuredto execute instructions for performing operations comprising: receiving,by a client device, a plurality of photograph filters, each of theplurality of photograph filters being configured to overlay an image toaugment the image, wherein a first filter of the plurality of photographfilters that is configured to modify an entirety of the image slidesacross the image in response to a first user input comprising a firstswipe motion, and wherein a second filter of the plurality of photographfilters that is configured to modify only a first particular feature ofan object depicted in the image slides across the image modified by thefirst filter in response to a second user input comprising a secondswipe motion received after the first user input, a given one of theplurality of photograph filters comprising weather information at a timeat which the image was captured by the client device, wherein theplurality of photograph filters is selected to be received by the clientdevice using bids from vendors, where the bids from the vendors arebased on characteristics of a user of the client device, a location ofthe user, and content of the image.
 2. The system of claim 1, whereinthe plurality of photograph filters is selected by a server based upon ageolocation of the client device on which the image is presented and theobject depicted in the image.
 3. The system of claim 1, wherein theweather information comprises a temperature at the time at which theimage was captured, and wherein the plurality of photograph filters isconfigured to be independently alternatively selectable by a user tooverlay the image.
 4. The system of claim 1, wherein a selected one ofthe plurality of photograph filters and a photograph corresponding tothe image are routed by a server to another client device.
 5. The systemof claim 1, wherein the first filter includes an object specific filter,and wherein the second filter includes textual content that is excludedfrom the first filter.
 6. The system of claim 1, wherein a serveridentifies when the client device presents the image on a display, andwherein the second filter causes a variety of filters with differentvisual attributes of the first particular feature of the object to besupplied to the client device.
 7. The system of claim 1, wherein thegiven one of the plurality of photograph filters includes brand indicia,and wherein the given one of the plurality of photograph filtersincludes an image associated with the brand indicia.
 8. The system ofclaim 1, wherein the first filter that overlays the image is replaced bythe second filter in response to user input to cause the second filterto overlay the image, wherein a third filter of the plurality ofphotograph filters that is configured to modify a second particularfeature of the object depicted in the image is selected in response to athird user input received after the second user input, wherein the thirdfilter includes textual content that is excluded from the first andsecond filters.
 9. The system of claim 1, wherein the plurality ofphotograph filters is configured to be independently selectable inresponse to a gesture applied to the image as presented on the clientdevice, wherein the plurality of photograph filters is selected by aserver based on popularity.
 10. The system of claim 1, wherein the givenone of the plurality of photograph filters includes a brand associatedwith an establishment proximate to a geolocation of the client device.11. The system of claim 1, wherein an attribute of the client device isidentified based on established preferences of the client device, andwherein the plurality of photograph filters is selected by a serverbased upon one or more attributes of the client device and one or moreattributes of the image.
 12. The system of claim 11, wherein theestablished preferences are derived from prior use patterns.
 13. Thesystem of claim 11, wherein the one or more attributes of the imageinclude a physical environment captured in the image.
 14. The system ofclaim 1, wherein the operations further comprise appending a message tothe image.
 15. A method comprising: receiving, by a client device, aplurality of photograph filters, each of the plurality of photographfilters being configured to overlay an image to augment the image,wherein a first filter of the plurality of photograph filters that isconfigured to modify an entirety of the image slides across the image inresponse to a first user input comprising a first swipe motion, andwherein a second filter of the plurality of photograph filters that isconfigured to modify only a first particular feature of an objectdepicted in the image slides across the image modified by the firstfilter in response to a second user input comprising a second swipemotion received after the first user input, a given one of the pluralityof photograph filters comprising weather information at a time at whichthe image was captured by the client device, wherein the plurality ofphotograph filters is selected to be received by the client device usingbids from vendors, where the bids from the vendors are based oncharacteristics of a user of the client device, a location of the user,and content of the image.
 16. The method of claim 15, wherein theplurality of photograph filters are selected by a server based upon ageolocation of the client device on which the image is presented and theobject depicted in the image.
 17. The method of claim 15, wherein theweather information comprises a temperature at the time at which theimage was captured, and wherein the plurality of photograph filters isconfigured to be independently alternatively selectable by a user tooverlay the image.
 18. The method of claim 15, wherein the given one ofthe plurality of photograph filters includes a brand associated with anestablishment proximate to a geolocation of the client device and anindicia associated with the brand, wherein the first filter includes afirst brand of a restaurant that sponsors the first filter and an imageof a food item associated with the restaurant, and wherein the secondfilter includes a second brand of a store that sponsors the secondfilter and an image of a tangible item associated with the store. 19.The method of claim 18, wherein the image of the food item in the firstfilter comprises an image of a hamburger or taco, and wherein the imageof the tangible item in the second filter comprises an image of a surfboard or sunglasses.
 20. A non-transitory machine-readable storagemedium that includes instructions that, when executed by one or moreprocessors of a machine, cause the machine to perform operationscomprising: receiving, by a client device, a plurality of photographfilters each of the plurality of photograph filters being configured tooverlay an image to augment the image, wherein a first filter of theplurality of photograph filters that is configured to modify an entiretyof the image slides across the image in response to a first user inputcomprising a first swipe motion, and wherein a second filter of theplurality of photograph filters that is configured to modify only afirst particular feature of an object depicted in the image slidesacross the image modified by the first filter in response to a seconduser input comprising a second swipe motion received after the firstuser input, a given one of the plurality of photograph filterscomprising weather information at a time at which the image was capturedby the client device, wherein the plurality of photograph filters isselected to be received by the client device using bids from vendors,where the bids from the vendors are based on characteristics of a userof the client device, a location of the user, and content of the image.